Papers with Small Language Model
Prompt Candidates, then Distill: A Teacher-Student Framework for LLM-driven Data Annotation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for data annotation use an aggressive approach prompting LLMs to determine a single gold label for each unlabeled sample. |
| Approach: | They propose a teacher-student framework that distills candidate annotations with a Small Language Model (SLM) they propose to use LLMs to generate and distill candidate annotation with slms to ensure unique labels are provided for downstream tasks. |
| Outcome: | The proposed method outperforms existing methods due to uncertainty in LLMs and is noisetolerant. |
It’s All About the Confidence: An Unsupervised Approach for Multilingual Historical Entity Linking using Large Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing approaches to EL for historical texts require substantial training data or rely on domain-specific rules that limit scalability. |
| Approach: | They propose an unsupervised ensemble approach combining a Small Language Model and an LLM for historical EL. |
| Outcome: | The proposed approach outperforms state-of-the-art models on four established benchmarks in six European languages from the 19th and 20th centuries. |
HCRE: LLM-based Hierarchical Classification for Cross-Document Relation Extraction with a Prediction-then-Verification Strategy (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to cross-document relation extraction (RE) focus on identifying relations between head and tail entities from single sentence or document. |
| Approach: | They propose a hierarchical relation tree-based LLM-based hierarchic classification model for cross-document relation extraction (HCRE) based on predefined relations, the model can perform hierarchically classification level by level. |
| Outcome: | The proposed model outperforms existing baselines and validates its effectiveness. |